§ ; Dynamical Solution of the On-Line Minority Game 

(N 



O 
m 



a 



o 



A C C Coolen and J A F Heimel 

Department of Mathematics, King's College London 
The Strand, London WC2R 2LS, UK 



P^ ' Abstract. We solve the dynamics of the on-line minority game, with general types 

C . of decision noise, using generating functional techniques a la De Dominicis and the 

c/3 I temporal regularization procedure of Bedeaux et al. The result is a macroscopic 

dynamical theory in the form of closed equations for correlation- and response functions 
defined via an effective continuous-time single-trader process, which are exact in both 
the ergodic and in the non-ergodic regime of the minority game. Our solution also 
explains why, although one cannot formally truncate the Kramers-Moyal expansion 
i-rt ! of the process after the Fokker-Planck term, upon doing so one still finds the correct 

^J ' solution, that the previously proposed diffusion matrices for the Fokker-Planck term 

are incomplete, and how previously proposed approximations of the market volatility 
can be traced back to ergodicity assumptions. 



> 

O ■ PACS numbers: 02.50.Le, 87.23. Ge, 05.70.Ln, 64.60.Ht 

o 



1. Introduction 



The Minority Game (MG) 0] is an intriguing variation on the so-called El-Farol bar 

"c^ ' problem which aims to capture and understand in the simplest possible way the 

C ■ cooperative phenomena in markets of interacting traders. Its equations describe the 

^ • stochastic evolution in time of the selection of individual 'trading strategies' by a 

community of traders which operate in a simple market. The rules of this market 

O ■ are that each trader has to make a binary decision at every point in time (e.g. whether 

to buy or sell), and that profit is made only by those traders who find themselves in the 

/\^ • minority group (i.e. who find themselves buying when most wish to sell, or vice versa). 

c^ ■ The essence of the minority game is that each trader individually wishes to make profit, 

but that the net effect of his/her trading actions is defined fully in terms of (or relative 

to) the actions taken by the other traders, and that there is a high degree of frustration 

since it is impossible for all traders to be successful at the same time. In spite of its 

apparent simplicity, the minority game has been found to exhibit non-trivial behaviour 

(e.g. phase transitions separating an ergodic from a highly non-ergodic regime) and 

to pose a considerable challenge to the theorist. Its (stochastic) equations do not obey 

detailed balance, so there is no equilibrium state, and understanding the model properly 

inevitably requires solving its dynamics. An overview of the literature on the MG and 

its many variations and extensions can be found in 0. 
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In the original minority game, the information supphed to the agents upon which 
to base their trading decisions consisted of the history of the market. However, it was 
reahzed |^ that the dynamics of the MG remains largely unaltered if, instead of the true 
history of the market, random information is supplied to the agents; given a (the relative 
number of possible values for the external information), the only relevant condition is 
that all agents must be given the same information (whether sensible or otherwise). 
This led to a considerable simplification of theoretical approaches to the MG, since it 
reduced the process to a Markovian one. In f^ it was shown that the relevant quantities 
studied in the minority game can be scaled such that they become independent of the 
number of agents A^ when this number becomes very large, opening up the possibility 
to use statistical mechanical tools. An interesting generalization of the game was the 
introduction of agents' decision noise |P], which was shown not only to improve worse 
than random behaviour but also, more surprisingly, to be able to make it better than 
random^. The studies and [|10[ finally paved the way for a number of papers aiming 



to develop a solvable statistical mechanical theory. 

Early theoretical attempts focused on using additive decision noise to regularize 
the microscopic stochastic laws and derive deterministic (non-linear) continous-time 



equations, which minimized a Lyapunov function |TT], |T2] . This approach was remarkably 
successful in identifying the location of a phase transition and in describing correctly 
some aspects of the behaviour of the minority game above this phase transition. 
However, it became clear later that the deterministic equations were only approximate, 
even for A^ — > oo (since they neglected relevant fluctuations). This initiated a debate 
about how to construct exact continuous-time microscopic laws for the minority game 
I3| , |T^, |1^, |1^, |1^, in which all participants restricted themselves to either deterministic 



or Fokker-Planck equations (some dealing with both additive and multiplicative decision 
noise [15|) but without agreeing on the expression to be used for the diffusion matrix. 



and all involving different types of approximations already at the microscopic level. 



Finally, in [pO| , ^ the problems relating to the continuous-time limit were circumvented 
by redefining the equations at the microscopic level directly in terms of full averages 
over all possible values of the external information, and the dynamics of the resulting 
so-called 'batch minority game' was solved exactly using generating functional methods 
|T9| (first without decision noise |Q, then also for general types of decision noise pl| ). 



In this paper we solve the dynamics of the original (on-line) minority game, along 
the lines of pO| , pT| (i.e. using generating functional techniques). We show how the 



problems and uncertainties relating to temporal regularization, which appear to have 
been responsible for generating the debates and approximations surrounding the proper 
form of the continuous-time microscopic laws, can be solved and resolved in an elegant 
and transparent way by using the (exact) procedure of |TS[ for deriving a continuous- 
time master equation without using the thermodynamic limit. This allows us to write 
down the full stochastic microscopic equations (without truncation or approximation) 

I Using a phenomenological theory for the volatility, based on so-called ' crowd- anticrowd' cancellations 
, this effect was partially explained in [|[ |j . 
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and solve the model by adapting to the present continuous-time (on-line) process the 
generating functional techniques which were employed successfully for the discrete-time 



(batch) process in [^, 21 1. The end result is an exact dynamical macroscopic theory, 
for general types of decision noise (additive, multiplicative, etc), in the form of closed 
equations for correlation- and response functions which are defined via an effective single- 
trader process, from which the statics follows as a spin-off. Our equations describe both 
the ergodic and the non-ergodic regime of the minority game, including transients. 

The full exact dynamical solution now available also allows us to make rigorous 
retrospective statements about the validity or otherwise of the various approximations 
proposed in the past, and to explain under which conditions such approximations could 
indeed have led to correct results. More specifically, we (i) show why it is in principle 
not allowed to truncate the Kramers- Moyal expansion of the microscopic process after 
the Fokker-Planck term (let alone after the flow term), but why upon doing so one can 
still find the correct macroscopic equations (for the present version of the MG), (ii) 
confirm that the different diffusion matrices for the Fokker-Planck term in the process, 
as proposed earlier by others, are incomplete or approximate, and (iii) indicate how 
previously proposed approximations involving the market volatility can be traced back 
to assumptions relating to ergodicity (and are thus valid at most in the ergodic regime). 

2. Definitions 

2.1. The Minority Game 

The minority game describes the dynamics of decision making by A^ interacting trading 
agents, labeled with Roman indices. At each round i of the game, each agent i has 
to take a binary trading decision Si{i) G {—1,1} (e.g. whether to sell or buy). At 
each round all agents are given the same external information J^(£) (representing e.g. 
the overall state of the market, political or economic developments, etc.), which here is 
chosen randomly and independently from a total number p = aN of possible values, i.e. 
fi{i) G {1, . . . , aN} for each £. To generate trading decisions, each agent i has S fixed 
strategies Rja = {Rja, • • • ) Rfl^) ^ {~1) 1}°^ at his/her disposal, with a G {1, . . . , 5*}. 
These strategies act as manuals (or look-up tables) for decision making: if strategy a is 
being used by agent i at stage i in the game, then the observation of external information 
fx{i) will trigger this particular agent into taking the trading decision Si{i) = i?^ . 
Hence the intrinsic dynamics of the minority game is not driven by the decision variables 
Si{i), but by the dynamic selection by each trader of trading strategies from his/her 
available arsenal, as described below. Each component i?^ is assumed to have been 
drawn randomly and independently from { — 1, 1} before the start of the game, with 
uniform probabilities. The strategies thus introduce quenched disorder into the game. 

Given a choice fi{i) made at the start of round i, every agent i selects the 
strategy di{i) which for trader i has the highest pay-off value at that point in time, 
i.e. ai{i) = arg max Piai^), and subsequently makes the binary bid bi{i) = Ri^.Ly The 
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(re-scaled) total bid at stage i is defined as A{£) = N~^/'^J2ih{^)- Next all agents 
update the pay-off values of each of their strategies a on the basis of what would have 
happened if they had played that particular strategy: 

pue+i)=pM-^R':l'^Aii) (1) 



The constant fj represents an (optional) learning rate. Note that the agents all behave as 
price takers, i.e. they do not take into account the impact of their own decisions on the 
total bid. We here consider only the S = 2 model, where the equations can be simplified 
upon introducing gi(£) = l[piiii)-pi2{^)], ^j = |[Rii + R.J2] and ^^ = i[Rji-Rj2]- Asa 
result, the selected strategy in round i can now be written as Riai(^) = <^i + sgn[gj(£)]^j, 
and the evolution of the difference is given by: 

V -f V V J V ,• 

with O = Ar-V2 ^ ^ ^ s)fja7V_ 



] 



2.2. Minority Game with Decision Noise 

The process (^ can and has been generalized in order to include traders' decision noise 

0, |11|. This can be done in many different ways; here we try to avoid being unnecessarily 
specific, and choose a general definition where we replace (as in e.g. pT|] ) 

sgnfe(£)] ^ a[q,{l),z,{t,] (3) 

in which the Zj{t) are independent and zero average random numbers, described by some 
symmetric distribution P{z) which is normalised according to fdz P{z) = Jdz P{z)z'^ = 

1. The function o"[g, z] is parametrized by a control parameter T > such that 
^"[Q')^;] G { — 1,1}, with \im.T^Qa[q,z] = sgn[q] and Hhht^oo Jdz P{z)a[q,z] = 0, so that 
T can be interpreted as a measure of the degree of stochasticity in the traders' decision 
making. Typical examples are additive and multiplicative noise definitions such as 

additive : a[q, z] = sgn[g + Tz] (4) 

multiplicative : a[q, z] = sgn[g] sgn[l + Tz] (5) 

In the first case (^) the noise has the potential to be overruled by the so-called 'frozen' 
agents, which have qi(t) ~ (jit for t — > oo (see [|3l)- ^^ the second case the decision noise 
will even retain its effect for 'frozen' agents (if they exist). These definitions represent 
situations where for T > a trader need not always use his/her 'best' strategy; for 
T — > we revert back to the process (0). The impact of the multiplicative noise (|^) can 
be characterised by the monotonic function 

A(r) = fdz P{z) sgn[l + Tz] (6) 

with A(0) = 1 and A(oo) = 0. For e.g. a Gaussian P{z) one has A(T) = erf[l/v^T]. 
We now find (|^) being replaced by 

g.(^+l) = qm - -^ if^A^^'^U^l z(£)], (7) 
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In this expression the quantity yl^[q, z] denotes the bid which would result upon 
presentation of information fi, given the microscopic state q of the system and given 
realization z of the decision noise: 

A'^[ci,z]=n^ + ^Y.^^a[q„z,] (8) 

ViV j 

3. Microscopic Probabilistic Description 

3.1. Temporal Regularization Using the Procedure of Bedeaux et al 

We convert the discrete-time on-line stochastic process (|I]) into an explicit Markovian 
description in terms of probability densities. For the present noisy version of the game 
one finds a microscopic transition probability density operator W^(q|q') which involves 
an average over the random numbers {^j}, indicated by {. . .)z'- 

p,+i(q) = /rfq' IV(qlq')pKq') (9) 



W^(q|q') 




9.-?: + -^e^1q',z] 



(10) 



In order to carry out systematically the temporal coarse-graining and transform the 
dynamics to an appropriately re-scaled continuous time t, we follow the systematic 
procedure of |1^ and define the duration of each of the iteration steps to be a continuous 



random number, the statistics of which are described by the probability iTiit) that at 
time t precisely i updates have been made. Our new process (including the randomness 
in step duration) is described by 

pt{q) = ^7r,(t)p.(q) = T.Mt) fdq! PF'(q|q')Po(q') 

i>Q i>0 

and time has become continuous. For 7r^(t) we make the Poisson choice irelt) = 
|f(t/A^)^e-*/^'^. From {i)^ = t/Z^N and {f)^ = t/A^ + t'^/A% it follows that An is 
the average duration of an iteration step, and that the relative deviation in £ at a given 
t vanishes for A^ — > as J {1"^)-^ — {^)1-/{^)tt = J A^ /t. This introduction of random 
step durations thus only introduces uncertainty about where we are on the time axis, 
which will vanish at the end of the calculation, provided we ensure liniAr^oo A^ = 0. 
The properties of the Poisson distribution under temporal derivation lead to 

|pt(q) = ^ [Jd^ w^(q|q')p*(q') -p*(q)} (n) 

3.2. Canonical Temporal Coarse Graining 

To find the appropriate scaling with A^ of A^ (and investigate the relation with the 
Fokker-Planck approximations of |T5[ and [|T^ in a subsequent section) it is instructive 



to expand the master equation ( p^ in powers of the learning rate: 



dt AnP ^1 J VN 
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E [Li Pt] (q) 



6 

(12) 



with 



[Li P] (q) 



T/* 



ni,...,njv>0 ^ 



^,E."^ 



9^ 



,n^!agr-..5g7 



X <^ p(q) 



7r^E(err--(e^r(^"[q'^]')= 



(13) 



From the i = \ term in this so-called Kramers-Moyal expansion one reads off the 
canonical scaling for the temporal coarse-graining time-scale Aa? (in order to guarantee 
a proper N ^ oo limit): 



[Li P] (q) 



V 



d 



«A.iV?%^^(^) 



Ir" 



1 



N N 



E^i-^i(^te'^])^ 



We are automatically led to the choice A^ = 0{fi/N) (this gives, for T = and for 
N ^ oo, as our first term exactly the batch dynamics studied in |2^), and we must 
require limAr^oo v/N = in order to guarantee limAr_^oo Aat = 0. We now put 

A^ = f^/2aN (14) 

(thereby en passant absorbing an additional distracting factor 2a, to find simpler 
equations later) and find the various terms (|r3|) in the Kramers-Moyal expansion (^) 
reducing to 



[Le P] (q) 



p(q 



E 



'^.E.«« 






M=i 



(15) 

Note that, according to ([I^), the present canonical definition of the time unit t implies 
temporal coarse-graining over 0(1/Ajv) = O^N/fj) iteration steps. 



3.3. Canonical Scaling of the Learning Rate 

The remaining freedom one has is in the choice of the learning rate fj. In order to 
find the appropriate choice(s) for the scaling with N oi fj we work out the temporal 
derivatives of the probability density Pk{q) = {S[q — qk]) of individual components of 
the microscopic state vector, where (/(q)) = Jdq p(q)/(q) (via integration by parts in 
the various terms of (|l^)): 

'^'~' ^' ^S[q-qk]tiekY{A^[q,z]%) 



"pM = i:^" 



dt 



e> 



\ m/H\ dq^ 



d_ 
dq 



2^[g - qk] 



-^ + ]YE^fc-^i(^fe.^j-])^ 
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Qk]^j:irkA'h^]]\ 



'N 



M=i 



2 fj 



(16) 



e>2 
We see explicitly that the diffusion term in this equation is of order 0{fj). Hence, 
in order to prevent the system from being overruled by fluctuations we have to choose 
fj = 0{N^) (with the fluctuations in individual components vanishing altogether as soon 
0). Following 



as lim 



TV^oo V 



11 1 and subsequent papers we choose r^ to be a constant 



which is independent of A^ in the remainder of this study. As a consequence we find 
that for A^ — > oo the single-trader equation (|T^) reduces to a Fokker-Planck equation, 
in agreement with [|15] and [|T^ (although, as we will show in a subsequent section, the 
diffusion matrices proposed in the latter two studies are both approximations). This, 
however, does not necessarily imply that the underlying A^-agent process can also be 
described by a Fokker-Planck equation. 



4. Properties of the Kramers-Moyal Expansion 

4.I. Relevance of Higher Orders 

With the definitions An = fj/2p and fj = 0{1) our full microscopic master equation 



ri|) becomes 
d 



|p.(q)^|/<*q'Mq') ii:(^ 



1 P 



q-q 



V 



^M'^[q',z 




The Kramers-Moyal expansion takes the final form 
d 



Pt(q) = J2 \Li Pt] (q) 



where 



dt 



[LiP] (q) 



:i8) 



e>i 



2fi 



~e-i 






If Z^ Z^ 

f^ rai,...,raiv>0 



(en •^- • (e^) 



■n-N 



Qi 



dq^-.-dq^ir 



or, equivalently. 



\Up\ (q) 



2?7 



~i-\ 



i\ 



E 



-ye— 



{p(q)(A^[q,z]Oz} 



(19) 



(20) 



Although for N —>■ oo the individual components of the state vector q were seen to 
have Gaussian fluctuations ([T^), the cumulative effect on the dynamics of the higher 
order {i > 2) terms in the expansion (|T^) (including cross-correlations) cannot simply 
be neglected, since also the number of components qi diverges with A^. This can be seen 
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more clearly in (^0]) than in (jl^). If we work out the evolution of averages of observables 
/(q), via integration by parts, we find that (p!8| , |20D predict 



4</> 



2E^E((-4"|q,^l'>. 



e>i 



i\ 



-ye— 



f 



For simple mean-field observables such as /(q) = N -^J^i CiQi"^ (with m independent 
of A^) one has [^Ei^^Yf = 0{N-^^^), and hence the ^ > 2 terms in the 
expansion are of vanishing order. In contrast, for observables which are dominated 
by fluctuations, this need no longer be the case. For instance, in the present system 
(with its overall reflection symmetry) one has /(q) = -4^ Y.i CiQi"^^'^ = 0{N'^); now 
[^Ei^^Yf = 0{N-'^^-^y^), and also the £ = 3 term could contribute to leading 
order. Moreover, there is no practical need to truncate the expansion, since (as we will 
show below) within the generating functional formalism it is perfectly natural to derive 
an analytical solution of the dynamics with all terms of the Kramers-Moyal expansion 
included. 



4-2. Explicit Form, of Flow and Diffusion Terms 

Let us now, by way of illustration, work out the first few terms of (^0]) for additive noise 
with P{z) = lK[l - tanh^{Kz)] (as in e.g. [0, |], with K such that Jdz P{z)z^ = 1), 
for which {a[q,z]), = tanh[/3q] (/3 = K/T): 

d 



[Li P] (q) = E ^ ^ 2p(q) 



^^'" + ^E^rC,tanh[/?g, 



[^2p](q) =vY. 



d^ 



Tt d(i^d(i3 



p(q) 



N 

1 

iV 



A^ 



T.ein^'i'^M')^ 



(21) 
(22) 



Equation (|^) describes, as expected, two types of fluctuations: [L2 p](q) = 
Xlfc^c^l^ {p(q)[M^ + M^] j-, one describing fluctuations induced by the randomly and 
sequentially drawn external information: 



rj ^ 



1 



and a second one describing fluctuations induced by the decision noise 

p 
— _!/_ ' 



^- = 1^ E ee E(ef )'(1 - tanh2[/5g.]) 



(23) 



(24) 



M=l 



The corresponding expressions for multiplicative noise are obtained by simply replacing 
tanh[/3gfc] -^ ^iT)sg'n\qk\ in equations (|2T|j23|j2^ ) . For both noise types the second 
contribution (|^) to the diffusion matrix vanishes in the limit /5 ^ cxd of deterministic 
decision making. 

§ Note that lim;3_>oo M^ = 0, so that the {Af^} represent the fluctuations due to external information 
selection in the absence of decision noise. For /3 < 00, however, the above causal/interpretational 
separation is not perfect, since the two sources of randomness will inevitably be intertwined. 
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5. The Generating Functional 

5.1. Definition and Discretization 

Instead of working with the Kramers-Moyal expansion, it will be more convenient and 
safe to return to the underlying master equation (0), which can be written as 



P fM=l J 



X ^lY.{e''^^'''''' '^'1^-'^} (25) 

where we have introduced auxiliary driving forces 6i{t) in order to identify response 
functions later. At this stage we discretise our continuous time, t —^ iS {i = 0,1,2, . . .) 
with intervals < 5 -C 1 which can be sent to zero independent of the limit N -^ oo 
(since the procedure of ||18|, as followed here, leads to a continuous-time master equation 
for any A^), so that the probability density of finding a path of microscopic states 
{q(0), q(5), q(2(5), . . .} can be written as 

Prob[q(0), q{5), q(25), . . .] = po(q(0)) U Wt[qit)\q{t - 5)] (26) 

i>0 

with 

X {^lT(e''l^^^^^O^'^-''% + (1 - M)| (27) 

The moment generating functional for our stochastic process is defined as 

Z[ijj] = { e^EtE.'^^W^^W ) 

l[{Mt) Wt[q{t + 6)\qit)]} po(q(0)) e^ E.E.'^^ (*)<?- W 



where, as in |2^, we introduced external forces 6i{t) to generate response functions 
later. Derivation of the generating functional with respect to the conjugate variables t/j 
generates all moments of q at arbitrary times. Upon introducing the two short-hands: 



Wt 



j:um, xr = ^E^^wef, 



N, VN 



with Si{t) = a[qi{t), Zi{t)], as well as Vq = Yln[dqi{t)/\/2TT], Vw = Yl^tldwt/V^n] and 
Px = Ylnt[dXf / y/27i] (with similar definitions for Dq, Vw and "Dx, respectively), the 
generating functional takes the following form: 

X |PqPqpo(q(0)) (e-tii:..^^ E.K^^W+^r^^Wl)^ 

X e*I^t«['?'W('?»{t+5)-g»{t)-e»{t))+»/'»(i)'?»{t)] ("28) 



ifiw'7 ^ x'v , 

e v^ ^ v^J — 1 
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We note that, as 6 ^ (and using SJ2t = 0{6'^)): 

y y 2s_ 

^L^ fj. ^— /t V 



n{-f?. 



Hence, if we choose 6 <^ A^~^ we obtain 



V2 V2'-iJ^o{SN) 



z[M 



VwDwDyiDit e 



M'^w'^+ix'^x'^+fle'^"' '""+ j2^^^-l+o(N0)] 



"t "^t ^'-^t -^t 






X e 



10 



(29) 



5.^. Disorder Average 

At this stage we carry out the disorder averages, denoted as [■ ■ ■]dis, which involve the 
variables ^f = ^{R'^ - R'^) and fi^ = ^A^-^ Ej(^^i + Rj2)- For times which do not 
scale with A^ and for simple initial conditions of the form po(q) = UiPoiQi) one finds: 






X 



X 



(30) 



U' 



'A X^ 

fi 2— It 



irjujJ'ln^'H — i=\/V2 



V2" _i+o(AfO) 



'^T.r^^T.t\-^tUt)^S^^t^^(t)]^ 



dis 



We concentrate on the term in ( pUD with the disorder averages. These are similar to 
but generally more difficult than those calculated in [^ (to which they reduce only 
for r] ^ 0). As in |^ they automatically generate the dynamic order parameters 
Cw = N~^E^Si{t)s^{t'), Kw = N-^EiS^{t)qi{t'), and Lw = N-^E^qi{t)q^{t') and 
their conjugates: 



n 



■ ' 'idis 



n 



'dndn ,r2Q+f^Je'^'<'"'^^'''^-l+o(iV0)] 



277 



X 



n 


g V2N _ 


I 





^ r2/v2+^j<g.{t)+£^,(i)] 



/2N 



(l/V2-J2tl^tUt)+£t^r(t)] 



dis 



n 



' dndn ,^f,+2.^^[,-<["+^]/v^_y+„(^o) 



2n 

"4lV 2^i 



X e 



(ft+EJ<9»w+^^»wl)'+(S-EJ<9»(*)+^^»(*)l)' 
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= n 






i 27r 

X e-27vEjEj<9,(t)+xr^,(t)]]' 



X e 2 



Insertion into (|30D , followed by the isolation of the order parameters via (5-distributions 
(whose integral representations generate the conjugate order parameters) then gives 

Z[f] = f[VCVC] [DKVk] [VLVL] e^[*+*+f^+°(^")] (32) 

The o{N^) term in the exponent is independent of the fields {ipiit)} and {9i(t)}. Upon 
choosing an appropriate scaling with 6 of the conjugate integration variables (viz. 
{x,w) — > 6{x,w) and {C,K,L) -^ 6'^{C,K,L), to guarantee the existence of a proper 
6-^0 limit of the various time summations), the three relevant exponents in (^) are 
given by the following expressions: 

^ = i6^ Y.[Cu'Cu' + ku'Kw + Lu'Lu'l (33) 



$ = a log 



tt' 

+xtCtt,Xt,] 



fvwVwVxVx e-^^' Y.tA^^^u'^t'+'^^^^tt'^t'- 



X e' 



(34) 



JM . J 

1 12 

with the standard abbreviation of the Gaussian measure Du = (27r)~2e~2" du. 
The average (. . .)z has now been reduced to a single site one: {g[zi,Z2, . . .])z = 
JY[t[dztP{zt)] g[zi, Z2, ■ ■ ■]■ Following |2^ we have also introduced the short-hands 
^Q = Ilt[d'Qit)/V^]^ T^w = Ylildwt/y/^]-, Vx = Ylf[dxt/\/2TT] (with similar definitions 
for T>q, Vw and Vx). 

6. The Saddle-Point Equations 

6.1. Derivation of Saddle- Point Equations 



We can now evaluate ( |32|) by saddle-point integration, in the limit N ^>- oo, and provided 
the order parameter functions depend asymptotically on their two time arguments in 
a sufficiently smooth (i.e. A^- independent) way. We define Gu' = —iKu'- Taking 
derivatives with respect to the generating fields and using the normalisation Z[0] = 1 
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then gives (at the physical saddle-point) the usual relations 

1 



Cu'= lim ^E(^^W^^('^')) (36) 

Af— >oo iV ■^^ 
t 

d 



Gw= lim — Vt^-— -(si(t)) (37) 

Lw = (38) 

Putting ipi{t) = (they are no longer needed) and 9i(t) = 6.9 (t) then simplifies ( |35| ) to 
n= log fvqVqpoiqiO)) e-'^'^w^'^w^it') 

X {e'^T.t Qit r'^'^T'^'^ ~e{t)-5 J2t' K,,Ai')]-^s^ Ett' mcu'^it'))^ (39) 



in which now s{t) = a[q{t), Zt\. Extremisation of the extensive exponent \1' + $ + i7 of 
(^) with respect to {C, (7, K, K, L, L} gives the remaining saddle-point equations 

Cu' = (s(t)s(t'))* Gtt' = ^H^ (40) 

^''-^do;: ^''-^di^ ^"'-^sz;; ^^^^ 



The effective single-trader averages (. . .)^, generated by taking derivatives of (pSf), are 
defined as 

, .. a . _ m{M[{q,s}]f[{q,s}]). 

inu^snu- jj^q(^M[{q,s}]). ^^'^ 

M[{q,s}] =po(g(0)) e-*^'S"'^W^"'^(*') fvq e-i^'Eu'«W4*'9(i') 

X ^isEtm['-^^^^^^-m-sj:^,k,,At')] (43) 

Upon elimination of the trio {C, K, L} via ( PD we obtain exact closed equations for the 
disorder- averaged correlation- and response functions in the N -^ oo limit: equations 
(^), with the effective single trader measure (^31). 

6.2. Simplification of Saddle-Point Equations 

The introduction of on-line evolution and decision noise into the dynamics has affected 
the terms $ (Q) and Q (0), compared to the analysis in pO|. In order to proceed we 
now have to work out the term $ (|33) further: 



$ = a log / DuVwVx 0[{a;, ti?}, u] 
where 



(l)[{x,w},u] = exp 






(45) 
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Upon expanding the 'inner' exponential in this expression in a power series, we can 
write (j) as an average of the form: 

"1 



(j)[{x,w},u] = ( 
where {■)n is defined as: 

(/[ni,n2,...])n 



exp 



-ifjJ2r^tWt[u + Xt\ 



E 

ni,n2...>0 



n^sV'yhs] /hi,n2, 



(46) 



(47) 



with the Poisson distribution Pa[i] = ji^ ""o-^- The first two moments of the distribution 



into 



are given by (nt)„ = 25/f] and (nj)„ = (25/fi) + 25/fi. We now insert ( 
followed by integration over u, {x}, {x} and {w} (in precisely that order). This gives, 
with K = iG, with the diagonal matrix Eu' = ^vntSu', and with the matrix D whose 
entries are defined as Du' = 1 + Gu''- 

$ = a\og(l)et~^EDE] 



(modulo an irrelevant constant). Taking the derivative of $ with respect to the matrix 
elements {Ltf, Gu', Gu'}, followed by setting L — > 0, gives (using the causality property 
Gu' = for t < t', which guarantees that Det(I + EG) = 1): 



9$ 



(9$ 



lim 
lim „ 

L^O dGtt' 

lim 



i-o dGu' 



1 

-a 

2 



a 



[l+EGy^EDEil+G^E)-^ 



{1+EG)-^E 



tt' / n 



ft/ n 



According to (0) this gives for the conjugate order parameters: 

1 



Ltf 

Gtf 
Ktf 



-zaS 



tf 







aRft 



with 



1 



E = - {(1+EG)-^EDE{1+G^E)-^ 

0^ \ 



R 



1 
52 



'l+EGY'E 



(49) 
(50) 
(51) 

(52) 

(53) 
(54) 

(55) 
(56) 



6.3. Evaluation of Poisson Averages 

Finally, it turns out that in these latter two matrices (^,|^) the averages over the {rit} 
can be performed exactly, by using causality. We first deal with (|56D (which is simpler): 

1 



Rtf 



& 



Y.{-iY{[{egYe]u^ 



£>0 
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We note that each of the terms in the expansion gives factorised averages due to Gss' = 
for s < s' (using only that {ns)n = 25/ fj for any s): 

{[{EGYE]tt')n = ii^vY^^ J2 {'n'tGtsinsiGs:,s2ns2Gs2S3 ■ ■ ■'n's,_,Gs,_,t'ne)n 

Sl,...,Sl_l 

t>si>...>se^i>t' 

= S'^'Gl 
giving the simple result 

R=hl + SG]-' (57) 



We note that in calculating ( |57| ) knowledge of only the first moment of the {rit} 
distribution was required. 

Next we turn to the noise covariance matrix: 

^te = i E {-lY''''{[{EGYEDE{G^E)%,)n (58) 

" ee'>o 

Now, again due to causality, only averages of terms with at most two n-variables with 
the same time index can occur. We have to take into account the extra contributions 
coming from all possible time pairings. Using {n"^) = {ns)'^[l + fj/^S] we derive: 

{[{EGYEDE{GW'UsOn = T.Ds,s'MegYeUsA{egY'ei,,^)^ 



ilfi) ^ ^ E E ^sis'^ Gsosi ■ ■ ■ Gs,_ise Gs'^s[ ■ ■ ■ Gs'^,_^s 

so>--->se s'g>...>s'i^ 

X {nso---ns,ns'^...ns')n 



i«'" E E D.,,, ' ■ ''^^-^'-'' 



so>--->se s'g>...>s'^ 



'^ts 



X GsQsi ■ ■ ■ Gsi_^si Gs'^s[ ■ ■ ■ Gs' 



£ i' r 

r V 



so>...>se s'g>...>s'^ i=Oj=0 

X Gsosi ■ ■ ■ Gsi_,se Gs's' ■ ■ ■ Gs' c' (59) 



where the factor I]f=o Sj=o ^s^s' counts the number of pairings occurring in the two 
types of time arguments (i.e. those with primes, and those without). We note that to 
find expression ( pOj) knowledge of only the first two moments of the {ritj-distribution 
was required. 

7. The Effective Single-Agent Process 
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7.1. Discretized Single- Agent Process 



Upon inserting the results ( |52| , |53| , |5^ we now find our effective single trader measure 
M[{g, s}] of ( ^3|) reducing further to the following expression (modulo a constant pre- 
factor reflecting normalisation, which is independent of the decision noise variables 
{z{t)}): 

'q{t + 6)-q{t) 



xS 



9it)+a5Y,Ru'sit')-y^r]{t) 



^ 



(60) 



with Rtt' = 5 ^[I + 5G]^J (|57|) . This describes a single-agent process of the form 
q{t + 6)- q{t) 



e{t) - aSY.Ru'a[q{t'),z{t')] + v^r/(t) (61) 



^ 



Causality ensures that Ru' = for t' > t. The variable zt represents the original 
single-trader decision noise, with {z{t))z = and {z{t)z{t'));, = 6tt', and ri{t) is a 
disorder-generated Gaussian noise with zero mean and with temporal correlations given 

by® 



7.2. Continuous Time Limit 

We can now take the limit 6 —>■ and restore continuous time. The bookkeeping of 6- 
terms is found to come out right, with partial derivatives with respect to the perturbation 
fields converting into functional derivatives, with time summations converting into 
integrals, and with matrices converting into integral operators (with the usual convention 
^Itt' -^ 6[t—t']). Upon making the ansatz that our order parameters are smooth 
functions of time, we then lose the remaining microscopic variables {z{t)} in the 
retarded self-interaction term (which are automatically converted into averages over 
their distribution) and end up with the effective single trader problem 

^g(t) = e{t) - a fdt' R{t,t'){a\q{t'),z\), + ^r^{t) (62) 

The correlation- and response functions (^,0) are the dynamic order parameters of the 
problem, and must now be solved self-consistently from the following closed equations 

C{t,t')= {{a[q{t),z])A<yW),z])^)^ (63) 

G(t,t') = ^((^bW,^]).). (64) 

(for t 7^ t'). The brackets (. . .)j, now refer to the stochastic process ( |62D (which 
no longer involves the {z{t)}, only their distribution). Note that the correlation- 
and response functions are generally discontinuous at t = t', viz. C{t,t) = 1 and 
G{t,t) = (in our derivation we have used the Ito convention). For additive noise with 
P{z) = lK[l - tanh^ {Kz)] (as in e.g. fTT], §, with K such that Jdz P{z)z^ = 1), for 
instance, one has {a[q,z])z = ta.iah[Pq] with inverse 'temperature' /3 = K/T. While for 
multiphcative noise one finds: {a[q,z])z = A(T)sgn(g) with A(T) = Jdz P{z)sgn[l + Tz]. 
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What remains is to take the continuous time hmit in our expressions for the effective 
noise covariance and the retarded self-interaction kernel. The latter, defined by (|57|), 
simply becomes 

R{t,t') =^m^[l + SG]-} 

= 6it-t') + j:{-^YG\t,t') (65) 

with the usual definition for multiplication of the continuous time kernels G^^^{t,t') = 
Jdt" G^{t,t")G{t",t'). The continuous time limit of the noise covariance kernel (^) 
becomes 

I i' r 



«'>0 so>...>Sf>OsJ,>...>s^>Oi=Oi=0 



V 

1 + (5o.,' — - 



X [1+G{si, 4)]G{so, si) . . . G{se-i, si)G{s'f^, s[) . . . G(4-i> s[) 

a'>o ° i=oi=o '- ^ -' 

X [1 + C(s^, 4)]G(so, si) . . . G(s£_i, s^)G(so, s'^) . . . G{s\,_^, s'^) 

(66) 

8. Stationary State in the Ergodic Regime 

In this section we study the long time limit of the effective single agent process ( |62D in 
the regime where the following three conditions are met: time translation invariance, 
i.e. 

limC(t + r,t) = C(r), lim G(t + r, t) = G'(r), (67) 

a finite integrated response (or static susceptibility), i.e. 

lim \dt!G{t, t')=x<oo (68) 



I.e. 



and weak long-term memory [^ 

lim / dt' G{t, t') = for any fixed t^ (69) 

Together these three conditions ensure that also the retarded self-interaction R and the 
noise covariance matrix S will become time translation invariant: limi_+oo R{t + t, t) = 
R{t) and linit^oo S(t + r, t) = S(r). We introduce the following notation for long time 
averages: / = limT-^oo t^^ Jo dt f(t). Upon assuming the above three conditions to hold, 
we can calculate the long time average for the single agent process (p^, giving: 

a 



dq/dt = e (a), + y/ari. (70) 

1 + X 



It has been noted and exploited earlier (in e.g. |T^ and pO|) that agents can be divided 



into two categories: frozen agents who get completely fixed on one specific strategy, and 
fickle agents who always continue to alternate their strategies. The first type of agent 
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will have a non-zero average preference velocity, dq/dt ^ 0, while for the agents in the 
latter group one has dq/dt = 0. For fickle agents it follows from equation (ffOD that 
{o')z = (1 + x)(^ + ^/aTj)/a. This implies that a necessary condition for an agent to be 
fickle is: 

fickle agents : \6 + \^fj\ < K^ii^O) -2^])^l —7- i'^^) 

Note that |((t[±oo, z])^! = A(T) for multiplicative noise, and that |((t[±oo, 2;])2| = 1 for 
additive noise. With the conventions A = A(T) for multiplicative noise and A = 1 for 
additive noise, we cover both cases by writing 7 = Aa;/(1 + x)- If ^^i agent is frozen, the 
preference velocity dq/dt must have the same sign as q{t), and thus also the same sign 
as {o')z- This implies 



a 



frozen agents : \6 + \/ari\ > \{<^)z\ = 7 (72) 

1 + X 

Since the two conditions (0)E1) ^^^ complementary, they are are not only necessary 

but also sufficient for characterizing agents as either fickle (^l]) or frozen (^2]). The 

asymptotic behaviour of the agent is thus completely determined by the persistent noise 

r], which is a Gaussian random variable with zero mean and a variance given by 

{V\ = lim y t dt' S(t') = -1±^, (73) 

t-^00 t Jo (1 + x) 

in which we find the persistent auto-correlation c = limT-_+ooT^^ Jq dtC{t). The latter 
observable can be expressed in terms of the integrated response x, the market's control 
parameter a, and the fraction of frozen agents 0. To do this we first separate the 
expression for c into frozen and fickle contributions, by using the conditions (|7T| , |72D (we 
may set 9 = 0): 

c = (i^'eHV^rJl - 7))^ + ((^'e(7 - Iv^rjl))^ 
= A^ {QilV^vl - 7))^ + (^) {fed - Iv^rjl))^ , (74) 



where denotes the step-function. Only Gaussian integrals remain (defining the 
distribution of fj) . To compactify the final result it is convenient to introduce 



/2 1 + c 



The fraction of frozen agents and the persistent correlation c can now be written as 
= 1- erf (y) (76) 

c = A^ Uy) + -J'^ x^'A . (77) 

V y^ Jo y/n J 

Under the conditions (^,^,^) we can calculate the static susceptibility without 
knowing the non-persistent parts of the response- and correlation functions. The reason 
for this is that, asymptotically, the frozen agents will not change their preferences when 
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subjected to an infinitesimal external perturbation field, whereas the fickle agents will 
react linearly to such a field. In formulae: 

Or, equivalently: 

The equations (ffSD, ( [771) and (|78D completely specify the stationary behaviour of the 
system. They are exact for the on-line minority game. We will not give a detailed 
exposition of the behaviour of the solutions of the equations here, because they have 
been presented and discussed at length in earlier papers. The interested reader is referred 
to [0 and |2^ in particular. 

However, it is appropriate at this point to discuss the status of our assumptions 



(^,^,^). Equation ([78|) tells us that x is positive and finite for a > 1 — 0, and will 
diverge at a = 1 — 0. Numerical solution of (^,|7^,^) shows that this happens at 
a = (yc(T), where ac(0) ~ 0.3374, and adT) tends to as T -^ oo (see [^ for the 
phase diagram in the (a,T) plane). For a > adT) simulations and theory are found 
to be in perfect agreement, and there is no evidence that the assumptions (^,^,|^) do 
not hold. Below a^T), however, the system's behaviour is known to depend strongly 
on initial conditions |[T5| , [20| , [T^ . This indicates that most likely not only condition 
(|68|) (finite integrated response) ceases to hold, but also ( |69l) (weak long-term memory). 
These two conditions may seem very similar; however, work in progress on a version of 



the MG where agents try to correct for their own impact on the market [Q indicates 
that it is possible to have a long-term memory and a finite integrated response |2^. For 
the original minority game as presented here we have found no evidence that this can 
happen. Hence, equations ([77|) and (|78D can for a > adT) be regarded as an exact and 



complete description of the persistent order parameters of the minority game. 

9. The Volatility 

9.1. Definitions and Exact Relations 

An important measure of the efficiency of a market is the average mismatch of buyers 
and sellers. In the case of the minority game this is given by the magnitude of the total 
bid A{i). Since in the present model the long term average A = limi^oo x Z]£<l ^(^) 
vanishes, the appropriate measure here is the volatility a, which measures the size of 
the fiuctuations of A (see equation (|D) in the stationary state: 

a' = hm jJ:A{i)' = hm jJ:{A>^(''>[ci{i),z{i)]y. (79) 

If the limit L — > oo is taken before the thermodynamic limit N ^ oo, then the volatility 
will be self- averaging with respect to the realization of the presentation of the patterns 
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{fi{i)}, with respect to the reahzation of the decision noise {z(£)}, and with respect to 
the reahzation of the quenched disorder variables ft and $. Hence, in the thermodynamic 
hmit, the average volatihty (over the above sources of randomness) will be identical to 
the single sample volatility. In the continuous-time version of the process, i.e. after the 
introduction of Poisson-distributed iteration durations, one may write: 

'1 



cr 



lim — 

r — >oo 7- 



dt 






(80) 



- dis 



Here the average without subscripts refers to the full stochastic process (including both 
the randomness in the selection of the external information as well as that induced by 
the decision noise). The approach followed in the present paper in fact allows us to 
study a more general object than the volatility: 



1 



-E[((^"[qW.^W]^"[q(0,z(t')]).)],,., 
p iM 



From (81 



E{t,t', 
the volatility follows as 
cr^ = lim - / dt E(t,t) 

T-»oo T Jo 



51) 



(82) 

In contrast to (|80D , the quantity (^) also describes transient bid fluctuations and 
their correlations. The quantity ( ^I]) is calculated via a simple adaptation of the 
corresponding calculation in ||20| (to which we refer for full details). One introduces the 
observables A^ = n^ + x^ /\/2 explicitly into the calculation of the generating functional 
using a delta function (written in integral form, which generates the conjugate variables 
A^), similar to the introduction of xf and w^. After having done the integral over Q^^ 
one then finds A^ = {u + Xt)/V^. If we apply this modification to equations (|4^ , |46|) , 
and perform the Gaussian integrals over u, x, x, w, w and A (in this order) we are left 
with (modulo an irrelevant constant): 



$ 




VAf" exp 



^ Af [(I + GEyD-\l + GE) 



At. 



From this it follows that the covariance matrix (RDI) is half the inverse of the matrix 
appearing in the above exponential between A^ and A^,. 

2«' = ^ ([(I + GE)-'D{1 + EG^)-\)^^ (83) 

In this expression one can carry out explicitly the average over the variables {^t}, 
similar to the calculation of the noise covariance matrix (|66D (to which ( ^T]) is found to 
be similar but, in contrast to the batch cases studied in [^, ^, not proportional). Upon 
subsequently taking the continuous time limit 5 — > this leads to an exact expression 
for the generalized volatility matrix (0), at any combination of times (t, t'), in terms of 
our dynamical order parameters C{t,t') and G{t,t'). 
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9.2. Expression for the Volatility in Terms of Persistent Order Parameters 



Since in practice it is very difficult to solve the order parameter equations ( |63| , |6^ for 
finite temporal separations, it would be helpful to find an expression for ( p3|) (and hence 
also the volatility) which involves persistent order parameters only. In the remainder 
of this section we discuss procedures to achieve this in the ergodic regime a > ac{T). 
First we turn to the long time correlation in bids. If t and t' are sufficiently separated 
and conditions (|67|j68|j69|) hold (so that G decays effectively on finite time-scales), the 



Poisson average in (|83D factorizes over the two E matrices, so that in the continuous-time 



limit the long term bid correlations reduce to 



H(oo) = lim lim H(t + r, t) 



;i + (5G)-iZ^(I + (5G^)- 



t+T,i 



lim lim lim - 

1 1 + C 



This result is exact. For the volatility itself, which in view of (|67| , |68|j69|) can be written 
as 0"^ = limj^oo S(t, t), and which depends on the detailed short-time structure of the 
kernels G and D, one has to resort to approximations. One such approximation is 
motivated by a property of mean-field disordered systems with detailed balance, where 
fluctuation-dissipation theorems (FDT) allow one to gauge away the non-persistent parts 
of the response and correlation function while leaving averages containing a single time 
index and averages containing infinitely separated times unchanged (see e.g. p6|); 
for equilibrium systems the resulting simpler equations are exact. Although no general 
analogous of equilibrium FDT's are as yet known for non-equilibrium systems such as the 
minority game, one could assume that the resulting recipe for removing non-persistent 
contributions to the various kernels also applies to non-equilibrium systems; this as yet 
ad-hoc assumption has recently been applied with remarkable success to a similar non- 
equilibrium problem [^. Here it would amount to removing all non-persistent parts 
of the various kernels, while retaining the relations b Yl,t Gtt' = X) ~ Y^kt Ct,t' = c and 
Gtt = 1, i.e. one inserts 

Gt,t' = c+{l-c)6u', Gt,t'=Xl{l-lSy~''-' (t>t'), (85) 

and takes 7 — »• at the end of the calculation. The only expressions containing G which 
will survive this limit, are those were each instance of G is summed over the whole 
history. More specifically, in the volatility 

^' = lj:j:{[iGEYD{EGr]io))^^ m 

contractions like b X^s GtsGt's cannot survive, as they are of order 7. Hence the Poisson 



average in (|86D factorizes over the E matrices, and the only non- vanishing term involving 



the diagonal part of D is the term i = i' = term. The resultfj]] is 



2 1 1+c , 1 



^ ~ -7 ^ + -(l-c). (87) 

2(l + x)2 2^ ^ ^ ^ 

Note: in the batch case pfl, El[ one finds a slightly different expression. 
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This expression for the volatihty depends only on persistent order parameters, and is 
independent of the learning rate fj. Work is in progress to explore and understand the 
theoretical basis, if it exists, of (or disprove the correctness of, as the case may be) the 
elimination of the non-persistent parts in general mean-field disordered systems without 
detailed balance, which underlies the derivation of ( |8^ ) from the exact expression (p^) 
in the ergodic regime. 

10. Relation Between Present Exact Solution and Previous Work 

The present study improves upon and generalizes previous work on the on-line MG in 
several ways: it is exact for A^ —;► cxo, it is a dynamical theory (with statics included as a 
spin-off), and it deals with a large family of decision noise definitions (including additive 
and multiplicative noise, and intermediates). In this section we compare the various 
approximations and assumptions made in previous studies with the exact solution, and 
also compare the solution of the on-line MG with that of the batch MG. 

10.1. Expressions for the Diffusion Matrix 

The first study in which one finds an expression for the diffusion matrix of the 
microscopic on-line process is |T^ (concerned with both additive and multiplicative 
noise). Comparison with the exact expressions ( ^^If ) shows that the matrix given in 
||T5| can be regarded as an approximation obtained by disregarding fluctuations due to 
the selection of external information, and retaining only those generated by the decision 
noise. Note that, similar to (0), it contains the factors (1 — tanh^[/5gj]) and vanishes as 
/3 — i> oo. The diffusion term presented more recently (for additive noise) in [^ can also 
be seen as an approximation of (|2^j2^ ), obtained upon replacing J2n^k^e{^^[''i, z]^)^ by 
^ij^T^ooT'^J2t<TJ2fj.^k^i-J2u{^'^[^.(f):^]'^)z: i-G- by neglecting certain correlations and 
subsequently introducing an additional temporal average over q. 

Note that neither of the two studies [jT5|, |1^] attempt to solve the dynamics; the 
objective of |T^ was to highlight the presence and role of the diffusion term in the 
microscopic equations (since at the time the microscopic laws were claimed to be 
effectively deterministic [111]). The authors of ||T^ have to restrict themselves to statics. 



since in their approximation and subsequent analysis they neglect the time dependence 
of the fluctuation term which they replace by the volatility (or, equivalently, they do not 
regard their volatility as time-dependent). However, their approach does shed additional 
light on the nature of the phase transition at a = adT) and the system state for 
a < ac{T). 

10.2. The Impact of Truncating the Kramers- Moyal Expansion 



Initially, in [|n|], the KM expansion was truncated after the Liouville term, leading to 



deterministic microscopic laws. In [0, |T7| it was truncated after the Fokker-Planck 
term. The question we turn to now is: what is the effect of such truncations, assuming 
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one would have taken the correct diffusion matrix (i.e. both terms p^ ) and (p^)) ? 
This question is even more relevant in view of the fact that, for additive noise, the 
exact equations (|75| , |77| , |78D , which give the stationary solution in the ergodic regime, are 



identical to those found earlier in [O 



If we trace back our derivations, we find that the effect of truncating the KM 
expansion after the deterministic term or after the Fokker-Planck term is to replace the 
function in (^6|) by 

0Det[{x,w},M] = e^S*™'("+^*), (88) 

or by 

respectively. It turns out that both expressions can be written in the form (^6|), i.e. 

(91) 



(/[ni,n2,...])„ = J2 

ni,n2...>0 



n^25/^K] f[ni,n2,...] 



but with alternative definitions for the statistics of the random variables {fit}: 

Deterministic : Pa[n] = 6[n — a] (92) 

^-■^{n~a)^/a 

Fokker-Planck : Pa[n] = (93) 

V27ra 

Both truncations can thus be seen as approximations of the true (Poisson) distribution 

which describes the noise due to random information selection: in the deterministic 

case one replaces the true P2s/fj [n] by a delta-distribution with the correct first moment 

(n) = 26 /fj, in the Fokker-Planck case one replaces the true P25/f^[n] by a Gaussian 

distribution and ensures that the first two moments (n) = 26 /fj and (n^) = {26 / fjY +26 / f] 

are correct. The above representation allows us to continue with our original derivation, 

in spite of the truncations, right up to and including the equations describing the 

effective single trader process ( |62| , |63| . |64|) , since the choice made for Pa[n\ only affects the 



kernels R{t,t') and S(t, t'). The derivation of expression (pSj), however, only involved 
the first moment of Pa[n] (see section 6), so both truncations would have led to the 
exact expression for R{t,t'). Surprisingly, the derivation of (^) only involved the first 
two moments of PaM, so whereas the deterministic truncation would have led to an 
incorrect expression for the covariance matrix S(t,t') (obtained by putting r^ — > in 
(|66|)), the Fokker-Planck approximation would have led to the correct expression (^) 
and hence to the exact dynamical order parameter equations. 

This explains why such truncations, although not a priori justified, here can lead 
to correct results. The deterministic theory would lead at most to correct equations 
in ergodic stationary states (where the learning rate of (^) drops out), but would 
fail to describe dynamical properties. The Fokker-Planck truncation would lead to the 
correct macroscopic dynamical theory, provided one uses the correct diffusion matrix, 
because the relevant order parameters of the present model fortunately turn out not to 
be sensitive to the (weak) non-Gaussian fluctuations. 
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10.3. Approximations of the Volatility 

Since the volatility ( |79D involves non-persistent order parameters (describing correlations 
over short temporal separations, even in the stationary state), it could not be calculated 
directly in any of the previous studies. Instead one had to resort to approximations. 

We repeat an argument here which was first given in [0, and which applies in the 
ergodic a > adT) regime. Upon assuming that phase space is sampled ergodically by 
the process, one may replace the time and pattern presentation averages by an average 
over the equilibrium probability measure: 



a 






(94) 



dis 



The decision noise average factorizes over sites, i.e. {a[qi,Zi]a[qj, Zj])^ = 

{a[qi, Zi])z{o'[qj, Zj])z for i ^ f (this is not an approximation). It was subsequently 
argued that in ergodic states the mean field character of the system means that also 
the ensemble average factorizes over sites, i.e. (cr[gj, Zi\a\qj, Zj\) = {a[qi, Zi]){a[qj, Zj]) for 
i y^ j- This should be regarded as an approximation, since, although for large A^ non- 
diagonal correlations can be assumed weak, they are also A^ in number and therefore 
cannot simply be discarded. The result of this mean field approximation is: 



cr^ ^ 



^E[((^^).)l.. + ^E^E[erer{i -((%.. ]).)^} 






dis 2 



1 



dis 



1 
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N 



E((^te'^])^)^ 



+ 0{N- 



-1/2N 



(95) 



dis 



which then leads, in the ergodic regime and for additive decision noise, more or less 

directly to 

, 1 1+c 1 

a ~ 

2(l + x)^ 

which is equation (pTl). This identification shows that the exactness or otherwise 



+ 5(i-c) 



(96) 



of the (traditional) approximation (p6| ) is crucially linked to the question of under 
which conditions non-persistent parts of dynamic order parameters can be 'transformed 
away' in ergodic non-equilibrium models (see section 9). It also emphasizes that 
such approximations must fail in the a < adT) regime, since in equilibrium systems 
such procedures are based on fluctuation-dissipation theorems and therefore typically 
reproduce the replica-symmetric solution. 

Numerical evidence presented in [^, |T2| shows that ( PBD is a very good 
approximation in the a > adT) region, until just above the transition point adT). 
It is not clear whether the slight discrepancy just above adT) is due to insufficient 
equilibration in the simulation, or because the approximation breaks down. 



10.4. Relation with the Dynamical Solution of the Batch MG 

Finally, comparison shows that the present dynamical solution (|62| , |63i |64D for the on-line 
MG can be regarded as a straightforward continuous-time equivalent of the discrete- 
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time dynamical solution of the batch MG [^, ^ , obtained simply by substituting time 
derivatives for discrete differences and integral kernels for matrices, for any type of 
decision noise. The only real difference is in the occurrence of the learning rate fj in 
(|66|) (which makes sense, since it reflects fluctuations relating to the random choice of 
external information, which are absent by definition in the batch models), which will 
be responsible for differences between batch and on-line MG models in the transients 
and in the non-ergodic region. This learning rate term, however, does not affect the 
stationary state solution for a > a^T), which explains why the stationary state of the 



batch models [^, |2l]] and the locations a^T) of their ergodicity-breaking transitions 



were identical to those found earlier (by others) for the on-line MG. 

11. Discussion 

In this paper we have solved the dynamics of the on-line minority game (MG), with 
general types of decision noise (including additive and multiplicative decision noise as 
specific choices). We have done so by following the successful approach which also 



recently led to the exact solution of discrete-time batch versions of the MG pG, 21 



(i.e. the application of generating functional techniques a la De Dominicis |T^), and 



we have dealt with the problems relating to temporal regularization, which occur only 
in on-line MG's, using the (exact) procedure of |18[. The end result is a macroscopic 
dynamical theory in the form of closed equations for correlation- and response functions 
(the dynamical order parameters of the problem) which are defined via an effective 
continuous-time single-trader process. These equations are exact for N ^ oo (where N 
denotes the number traders), and they incorporate all static and dynamic properties of 
the on-line MG, both in the ergodic and in the non-ergodic regime. 

We have used our theory to resolve a number of open problems related to 
approximations and assumptions made in previous studies. For instance, we show why it 
is in principle not allowed to truncate the Kramers- Moyal expansion of the microscopic 
process after the Fokker-Planck term (let alone after the flow term), but why upon 
doing so one can for the present version of the MG still find the correct macroscopic 
equations, we confirm that the different diffusion matrices for the Fokker-Planck term 
in the process as proposed earlier by others are incomplete or approximate, and we 
indicate how previously proposed approximations involving the market volatility can be 
traced back to assumptions relating to ergodicity. 

The macroscopic theory now available is not only exact, but also more 
comprehensive than its on-line predecessors: it deals with the full dynamics (with statics 
included as a by-product), and it generalizes the class of decision noise definitions. 
We also hope that our theory will end the discussions about which are the correct 
microscopic equations for the MG, and that it can be used in future as the canonical 
starting point for both analyses and generalizations (e.g. dynamics in the non-ergodic 
regime, or using the real market history as external information as originally proposed 
in m), and for approximations (for which there is still a need, since it is generally a 
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non-trivial task to solve our macroscopic laws). This might well include some of the 
approximations which have already been proposed earlier, which can now be provided 
with transparent interpretations and with a guide for systematic improvement. 
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